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(54) Decompression system for compressed video data for providing uninterrupted 
decompressed video data output 



(57) A Video decompression system (1 00) for de- 
compressing consecutive streams of compressed video 
daia lo provide a continuous, uninterrupted decom- 
pressed video data output stream comprises a controller 

(11Q), decompresston circuits (120), each circuit tiaving 
a compressed video data bulfer (121), a decoder (120), 
and a decompressed video data buffer (125). Eacii of 
the compressed Video data buffers (121) is coupled to 
an input switcti (1 05) and each of the decompressed vid- 
eo data buffers is coupled to an output switch (115), Si- 
mullaneous decompression in the circuits (120) results, 
when necessary, in each decompression circuit (120) 
having several Irames of decompressed video data 
available for Immediate display before tiiat decom- 
pressed video data is actually required. 
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Description 

Tills invention is in line field of video data and the 
provision of video data to users. In particular It relates 
to methods and apparatus for decompresing com- 
pressed video data. 

For purposes of this description, video date typicalfy 
comprises the video end audio data contained in a stored 
video program. However other data including but not 
(imited to text and graphics may be Included In the video 
data without in any way affecting the operation of the 
present Invention or the substance of this descrlptbn. All 
references herein to Video data shoiikJ therefore be con- 
sidered in the broadest sense. 

Video servers for providing video data to users are 
known. Although uncompressed video data can be 
stored in such servers and sent to users, the sheer 
amount of data In even a shori video program usually 
requires that the video data be stored and manipulated 
in a compressed form. Methods and apparatus for ac- 
complishing the compression and decompression of vid- 
eo data are known. 

One known compression format is sponsored by the 
Ivlotlon Picture Expert Group and Is known as MPEG, 
Although both an MPEG-1 and an MPEG-2 compression 
format are l4nown, their differences are not relevant to 
the present invention. 

rviPEG compfession is based on the fact lhat from 
one frame of video data to the next, there are compara- 
tively lew changes, even when objects or persons are in 
motion. It is therefore not necessary to store all ol the 
video data contained In each frame. Rather, after a base 
frame has been stored, each successive frame can be 
recreated by storing only the video data that describes 
objects or persons that have either changed or moved. 
Periodically, a complete frame o( video data must be 
stored to re-initialize the process. This type ol data com- 
pression is called motion compensalion- 

IVIPEG compressed video data consists of tnrcc 
types of frames. The first, an intra coded frarre (hence, 
an l-frame) provides all the video data needed to fully 
describe that particular frame. A predicted frame (hence, 
a P-frame) provides onfy information about how the 
P-frame differs from the last I- or P-frame. Finally, a bi-di- 
rectional frame (hence, a B-frame) provides information 
about how the B-frame differs from both the preceding I- 
or P-frama and the next l-or P-frame, The decompres- 
sion of the video data contained in a B-frame requires 
the decompression of two frames, either both l-frames, 
an l-frame and P-frame, or both P-frames, Decompress- 
ing a P-frame requires the video data contained in the 
preceding P- or l-frame. Typically, storing an l-frame re- 
quires three times as many bits as a P-frame, and storing 
a P-frame requires roughly three times as many hits as 
a B-frame. These relative storage requirements of the 1- 
B-, and P-frames are provided only for comparison pur- 
poses and no limitation of the present invention to the 
stated relative storage requrements should be implied. 



Fig, 1 shows the typical transmission and display or- 
der of a series of fulPEQ compressed Video data frames. 
As P-frames need the video data contained in a decoded 
l-frame to be decoded, and as B-frames need the video 

5 data contained in either or both a decoded l-frame and 
a P-frame, the transmission order of compressed IVIPEG 
frames differs from the display order of decompressed 
MPEG frames. Both Ihe transmitted and displayed 
framesbegin with an l-frame, and another l-frame occurs 

w roughly every fifteen frames thereafter. During transmis- 
sion, two B-frames are preceded by a P-frame, When 
displayed, afterthe first l-frame, two B-frames lolbw,and 
then a P-frame, As neither a B-lrame or a P-frame can 
be decompressed without reference to an l-frame, all 

'5 compressed video data streams must begin With an 
I -frame, 

A known architecture fordecoding MPEG videodata 
streams Is shown in Fig. 2. Decompression system 10 
consists of compressed video data bufier 11, decoder 

20 1 3, and decompressed video data buffer 1 7. An input bus 
15 provides a stream of compressed MPEG video data 
to buffer 1 i . As bus 1 5 provides video data at a fixed rate, 
some time elapses before enough video data is stored 
in buffer 11 for decoder 13 to begin decompressing the 

SB video data. 

A latency time exists before enough video data en- 
ters buffer 11 for decoder 13 to begin decompression. 
This latency time is herein called buffer filling latency 
time. An even longer latency time occurs due |o the na- 

30 lure ol the MPEG video data. Either an l-frame and a 
P-lrame, two P-lrames, or two l-frames must be decom- 
pressed and available before a S-frame can be decom- 
pressed. Typically two B-frames are transmitted after a 
P-frame. The system must receive and decompress 

35 these I- and/or P-frames before the B-frames can be de- 
compressed. The time required for this decoding and re- 
o dering is herein called a reordering latency time. 

The effect of the reordering latency lime Is noticea- 
ble every time a new video program begins. The reor- 
del ing latency time and the bulfer filling latency lime to- 
gether result in Ihe system generating several blank 
frames between the old and new video programs while 
the new video program is decompressed sufficiently for 
display In some known systems, as many as eight such 

-'5 blank frames are generated between two consecutive 
video programs. These blank frames are highly undsslr- 
abfe. 

.■^t p-9sent, no known system corrects this deficiency 

so A first preferred embodiment ol the present inven- 
tion comprises a video decompression system that can 
accept multiple compressed video data streams as input. 
For purposes of this description a compressed video 
data stream can be comprised of a single video program 

55 or multiple video programs. Different video data streams 
will ihereforc comprises different individual video pro- 
grams. The video data streams may be available simul- 
taneously at the input of the present Invention or they 
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can bG received at different times, Ttie system will de- 
compress and display a first video data stream Prior to 
the end of the first video data stream, the system vvili bs- 
gin accepting as Input and decomprBssIng another video 
data stream. Ai least several frames of the MPEG com- 
pressed video data comprising the second video data 
stream will be decompressed and avaslable lor display 
as soon as the first video dala stream ends. The buffer 
filling and reordering latency limes which occur when the 
present invention begins to dacomprass its first video 
data stream occur off-line, so the user has no direct ex- 
perience of it. As successive video data streams begin, 
their buffer tilling and reordering latency time occur while 
the previous video data stream Is being decompressed 
and displayed. The user does not experience these la- 
tencies either. In known systems, the user experiences 
both the buffer filling and the reordering latency times as 
blank frames between successive video data streams 
every time a new video data stream begins. 

In the preferred embodiment, the input switch con- 
trols the flow rate of two video data streams on two video 
data Input lines which comprise the inputs o' the p'e- 
ferred embodiment. Initially, the first video data stream 
flows into the present invention at a first rate. When (he 
second video dala stream Is later allowed to flow, il flows 
Into the system at a secwid rate on the second line, in 
the preferred embodiment, the first rate is higher than 
the second rate. In other embodiments, this need not be 
the case. The input switch can halt the flow of either 
stream. 

The input switch is in lum coupled to two decom- 
pression circuits, each circuit comprising a compressed 
video data buffer, a decoder, and a decompressed video 
data buffer. Incoming compressed video data is stored 
in the compressed video dala buffer, decompressed In 
the decoder, and stored temporarily in the decom- 
pressed video data buffer. The decompressed video 
data buffer from both circuits Is in turn coupled to an out- 
put switch. 

Both decompression circuits and the inpiii nrd o:i\- 
put switch are coupled to a microcontroller. Under in- 
structions from a stream scheduler, the microcontroller 
determines which decompression circuit will accept vid- 
eo dala al the fimt rate and which will accept it the second 
rate. The first decompression circuit receivesthefirst vid- 
eo data stream at the first rate decompress it, and pro- 
vides it to the output bus. 

Prior to the end ol the first video data stream, the 
system will instruct the input switch to cegin fiowing the 
second video data stream into the second decompres- 
sion circuit at the second fate. Several frames of the sec- 
ond video data stream wifl be decompressed and stored 
in the decompressed video data buffer before the first 
video data stream ends. When the first video data 
streams ends, the system immediately begins to display 
the decompressed second video data stream and simul- 
taneously increases the rate of video dala flow into the 
second decompression circuit to the first rate. In this 
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manner, successive video data streams are displayed 
without the user seeing any blank frames between the 
V deo data streams or experiencing the buffer filling and 
reordering latency times directly. 

5 Processing at first and second iBtes by alternate de- 
compression circuits continues indefinitely until all video 
data streams have been displayed. 

The preferred embodiment will now be described in 
detail with reference to the figures listed and described 

10 below. 

Fig 1 shows typical MPEG compressed frame 
transmission and decompressed frame display 
sequences(Prior Art); 

JS 

Fig. 2 shows a known system for decompressing 
video data (Prior Art); 

Fig. 3 is a block diagram of the preferred embodi- 
30 ment of the present Invention; and 

Fig. 4 is an example showing the relative rates of 
video data transmission through the first and second 
decompression circuits of the present Invention dur- 
ing typical use, 

A first preferred embodiment of the present inven- 
tion is illustrated in Fig. 3. Decompression syslem^ 100 
comprises input switch 1 05, first and second decompres- 

30 sion circuits 120 and 130, which in turn are further com- 
prised respectively of compressed video data buffers 
121 and 131, first and second decoders 123 and 133, 
decompressed video dala buffers 125 and 135, output 
switch 115 and microcontroller 110. Stream scheduler 

35 t50 is coupled to microconlroller 110. IWicroconfroller 
110 is a commercially available Motorola 68331 micro- 
controller and requires no further description herein. 

in each decompression circuK, Ihe compressed vid- 
oo cata buffer is coupled to the decoder. Which is in turn 

-iS coupled to the decompressed video data buffer. The de- 
coders in the preferred embodiment are commercially 
available STI 3500 MPEG decoders from SGS Thomp- 
son and require no further description herein. 

Input switch 1 05 is coupled to two compressed video 

45 data input lines, both compressed video dala buffers 121 
and 131, and to microcontroller 110. Output switch 115 
is similarly coupled to decompressed video data buffers 
1:25 and 135 and to nvcroconlrotler 110. Microcontroller 
1 1 0 is further coupled to both decoders 1 23 and 1 33, the 

so (wo compressed video data buffers 1 21 and 1 31 , as well 
as to server controller 1 50. Although microcontroller 1 1 0 
receives instructions from stream scheduler 1 50, stream 
scheduler 1 50 does nol form part ol the present inven- 
tion. 

55 Although shown as separate bulfers in Fig. 3, the 
compressed and decompressed video data bullers can 
be realized as a single buffer, which would be accessed 
in a known manner. The exact configuration of the buff- 
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srs can thGrefors be varied considerably from tine illus- 
trated embodimant wilfiout ciianging the present mven- 
tion in a material way. In the preferred embodimant, the 
bullers comprise eight 256K words by 16 bits dynamic 
random access memories ("DRAMs") coupled to the de- 
coder. The DRAMs are commercially available from 
many vendors, including Hitachi, and require no further 
description herein. 

In operation, two separate compressed video data 
streams enter input switch 105, Under command of mi- 
crocontroller 11 0, video data flows into one of the decom- 
pression circuits at a first rate and flows into the other at 
a second rate. For purposes of tills description only, and 
Without implying any limitation, video data will be as- 
sunr^ed to flow initially into decompression circuit 120 at 
a high rate and into decompression circuit 1 30 at a lower 
rate. It should be Understood that nothing tierein con- 
strains the second rate to be less than the first rate. Al- 
though the preferred first embodiment uses a first rate of 
15 megabits per second and a second rate of 7.5 meg- 
abits per second, these rates were chosen to reduce 
overall bandwichh demands. The second rate in other 
embodiments could be the same or indeed higher than 
the first rate. 

As compressed video data buffer 121 begins k> fill 
with the video data It is receiving at a ti^h rate, decoder 
123 begins decompressing that video data. Decom- 
pressed Irarties of video data are stored in decom- 
pressed video data buffer 125. After the first I- and 
P-frames have been decompressed and stored in de- 
compressed video data buffer 125, decompression of 
the B-frQmes that separated the I ^ and P frames begins. 
Decompressed video data framss are removed from de- 
compressed video data buffer 12S in the proper order 
under the direction of microcontroller 110 and sent 
through output switch 115 to an output bus. 

When the first compressed video data stream Is 
nearfy finished, the process of decompression begins in 
dGcomprassion circuit 120, Microcontroller 110 Instructs 
input switch 105 to begin flowing the second ccmpressed 
video data stream into decompression circuit 1 30, albeit 
at the second, lower rate. If the video stream being de- 
compressed in decompression circuit 120 Is very long, it 
is possible that decompression circuit 1 30 wili fill Its de- 
compressed video data buffer to capacity prior to the 
completion of the first video data stream, despite the mi- 
crocontroller only ordering the second decompressioi 
circuit to begin operation shortly before the first video 
data stream Is predicted to end. Microcontroller 110 will 
In that case instruct Input switch 105 to stop the flow of 
video data into decompression circuit 1 30. 

Once the video program flowing into decompression 
circuit 120 ends, decompression circuit 130 has several 
frames of video data decompressed and ready for im- 
mediate display or output. Output switch 115 would be 
Instructed by microcontroller 110 to switch the output to 
decompression circuit 130 as the output from decom- 
pression circuit 120 ends, Insuring a continuous gener- 



ation of decompressed video data without any biank 
frames between video streams. InptJl switch 105 also be- 
gins to flow the second video data stream into decom- 
pression circuit 1 30 at the higher, first rate of video data 

s flow, Inpulswitch 105 may provide decompression circuit 
120 with a low rate of video data immediately. btJt it Is 
more likely that no video data will be flowed into decom- 
pression circuit 120 forat ieasl some Inlervaloi tirr^e. This 
process would continue alternatively, with each decom- 

to pression circuit alternatively providing the output from 
the system. In this manner, the buffer filling and reorder- 
ing latency times are not experienced by system users 
and no biank frames occur between video programs. 
An example of the process of aitematlvely providing 

IB video data to decompression circuits 120 and 130 Is Il- 
lustrated graphically In Fig. 4. At limeT^ , decompression 
circuit 120 is receiving and decompressing a first video 
data stream at a high rate. Decompression circuit 130 Is 
not receiving video data. At time Tg, microcontroller 110 

20 has instructed input switch 105 to begin flowing the sec- 
ond video data siream into decompression circuit 1 30 at 
the second, lower rate At lime T^, the first video data 
stream being processed by decompression circuit 120 
ends. Immediately, decompression circuit 1 30 begins re- 

2S ceiving video data at a high rate, while simultaneously 
sending Its stored decompressed video data frames to 
the output bus. At time T4, microcontroller 110 instructs 
input switch 105 to begin flowing the next video data 
stream into first decompression circuit 1 20 at the second, 

30 lower rate. At time Tj, the bullars in decompiassion cir- 
cuit 120 are lull and video data flow to that circuit ceases. 
A! time Tg, the video data stream being decompressed 
by decompression circuit 130 ends. Decompression cir- 
cuit 1 20 now receives its next video data stream at the 

3B high, first rate, while providing Its stored decompressed 
video data frames to the output bus. Video data stream 
input to decompression circuit 130 ends until time T7J 
when microcontroller 110 again instructs input switch 
105 to begin liowirg the next video data stream into de- 

'fo cofnp:essicn circuil 1 30 atthesecond, lower rate. At time 
Tg, the video data stream being decompressed by de- 
compression circuit 120 ends and data stream input to 
that circuit is also ended, while simultaneously video 
data stream Input to decompression circuit 130 Is in- 

45 creased to the first, high rate and the stored frames of 
decompressed video data In decompression circuit 
1X's bufferare provided as output. This sequential proc- 
ess continues until all video data streams provided as 
Input have been decompressed and provided as output 

so in a continuous output stream. 

Although the present invention has been described 
In detail with reference to only two decompression cir- 
cuits, nothing herein should be taken to limit the present 
invention to only two such circuits. The expansion of the 

55 system to more decompression circuits would be a 
straightforward process and would provide even greater 
system flexibility and usefulness. In a system with more 
than two decompressbn circuits, the switching process 
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would obviously nof have to occur altDrnativsly or se- 
quentially. 

The present invention can bG used in a r=ai time d s- 
play system, where different video programs must be 
displayed consecutively without pauss. Video "clips" can s 
be playedfrom randomstarl points to random end points 
followed by another video "clip" with random star! and 
snd points, with no delay or blanfc frannes between the 
"clips". It should be noted that such random start points 
will require additional flexibility from the system, as de- io 
compression will have to begin early enough la decom- 
press the video data stream up to the "random" start 
point, as all preceding decompressed video data 
streams will have to be used to decompress the video 
data stream up to the chosen start point and then dis- 's 
carded. The presenl Invention Is also uaefuf In video data 
editing systems, where video data programs or portions 
of programs must be spliced together. Video program in- 
sertion, advertisement insertion and video editing are all 
environments within which the present invention would so 
be useful. 



Claims 

2S 

1. A system(IOO) for decompressing video data 
streams and for providing continuous video data out- 
put, ths syst6m(100) comprising: 

an input switch(IOS) coupled to a plurality of 
compressed video data Input lines, ttie swilch(105) so 
capable of selecting input Hnss and capable of con- 
trolling the video data flow rate o1 the selected input 
lines: 

a plurality of decompression modulas(120) 
coL^ledtothe input switch{105) for decompressing 3S 
compressed video data received from the input 
swkch (105) and storing decompressed video data; 

an output switch(1lS) coupled to the decom- 
pression modul6s(1 20), the output switch(1 1 5) cou- 
piir>g only one of the decompression modules(120) 40 
to an output bus at any time; and 

a controller(IIO) coupled to the input switch 
(105), the decompression modules(1 20), and the 
output switch(115) for selecting which decompres- 
sion module(120) will receive video data at a first 
predefined rate, the decompression mQdule(120) 
receiving video data at the first predefined rate also 
being Coupledto the output bus by the output switcli 
(115). 

2. The system(lOO) of claim 1 wherein the decompres- 
sion modufes(12a) not receiving video data at the 
first predefined rate subsequently receive video data 
at a second predefined rate. 

55 

3. The system( 1 00) of claim 2 wherein the second pre- 
deffned rate has a lower, the same, or a higher bit 
per second rate tlian the first predefined rate. 



The system (100) of any preceding claim, wherein 
the decompression modules further comprise: 

a buffer (121) for storing compressed video 

data; 

a decoder (123) for decompressing video 
data; and 

abuffer (121 or 125) for storing decompressed 
video data. 

The system (100) o1 any preceding claim, wherein 
the controller (110) commands the first switch (105) 
to halt video data flow at the second rate if the video 
data decompression module (120) receiving video 
data at the second rate cannot store any more 



A video decompression system (100) comprising: 

a first switch(1 05) coupled toal least two video 
data input lines, the first switch controlling the direc- 
lion and rate of video data flow from the video data 

input lines; 

at least two video data decompression arrays 
(1 20) coupled to ths first switch(1 05), the video data 
decompression arrays(120) storing compressed 
video data, decompressing the stored compressed 
video data, and storing the decompressed video 
data; 

a second swltch(115) coupled to the video 
data decompression arrays(i20) and to an output 
bus, the second switch(115) directing output from 
the at least two video data decompression arrays 
(120) to the output bus; and 

a controller(IIO) coupled to the first switch 
(105), the video data decompression arrays(120), 
and to the second switch(11 5) for controlling the flow 
of video data through the aystem(1 00). 

The system of claim 6, wherein the controller (110) 
commands the first switch (105) to provide video 
data to the first video data decompression array 
(1 20) at a first rate and to provide video data to the 
remaining video data decompression arrays (120) 
at a secorjd rate a predetined period of time after the 
first video data array (120) begins receiving the 
video data at the first rate. 

The system of claim 7, wherein the controller (110) 
commands the first switch (105) to halt video data 
flow to the first video data decompression array 
(120) after a video data stream being decompressed 
by the first video data decompression array (120) 
has ended, the controlfer (110) also commanding 
the first switch to provide video data io a second 
video data decompression array (120) at the first 
rate after the first video data stream has ended. 

The system of claim 7, wherein the controller (110) 
commands the second switch (115) to couple the 
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first video data decompression array (120) to ttie 
output bus whiie tiie first video data decompression 
array (120) receives video data input attfie first rate. 

10. A metliod for providing continuous decompressed s 
video data to an output from a plurality of com- 
pressed video data streams, ttie compressed video 
data streams, tiie metfiod comprising the steps of: 

receiviag tfie plurality of compressed video 
data streams; lo 

selecting a Ilrst video data stream; 

decompressing tiie selected first video data 
stream at a first rate for the duration of the first video 
data stream: 

decompressing the remaining video data 
streams at a second rate prior to the end of the first 
video data stream; 

providing the decompressed first video data 
stream to the output; and 

repeating the selecting, compressing, deconv so 
pressing and providing steps until all the video data 
streams have been provided to the output. 
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